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Recent research has suggested that introducing a disfluency in the context of written composition (i.e., 
typing with one hand) can increase lexical sophistication. In the current study, we provide a strong test 
between two accounts of this phenomenon, one that attributes it to the delay caused by the disfluency 
and one that attributes it to the disruption of typical finger-to-letter mappings caused by the disflu¬ 
ency. To test between these accounts, we slowed down participants’ typewriting by introducing a small 
delay between keystrokes while individuals wrote essays. Critically, this manipulation did not disrupt 
typical finger-to-letter mappings. Consistent with the delay-based account, our results demonstrate 
that the essays written in this less fluent condition were more lexically diverse and used less frequent 
words. Implications for the temporal dynamics of lexical selection in complex cognitive tasks are 
discussed. 

© 2016 Elsevier B.V. All rights reserved. 


“That’s not writing, that’s typing.” 

[Truman Capote] 


1. Introduction 

Recent research has suggested that in some circumstances, 
introducing a disfluency can benefit performance (e.g., Ball, Klein, 
& Brewer, 2014; Mueller & Oppenheimer, 2014). A surprising 
example of this general phenomenon was recently reported in 
the context of written composition by Medimorec and Risko 
(2016), who found that decreasing transcription fluency (or typing 
speed) by having individuals type with one hand resulted in 
more lexically sophisticated essays. This finding is surprising 
theoretically because transcription fluency is typically thought to 
be positively correlated with writing quality as the automatization 
of transcription arguably affords the re-distribution of resources to 
higher level writing processes such as planning (Fayol, 1999; 
Kellogg, 1999). Thus understanding how transcription disfluency 
influences aspects of writing provides a unique opportunity 
to gain a deeper understanding of the basic mechanisms underly¬ 
ing lexical selection in written composition (and language use 
in general). Here we provide a strong test between two potential 
theoretical accounts of the effect of disfluency on lexical 
sophistication. 
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3.1. Typing disfluency and cognition 

In Medimorec and Risko (2016), transcription disfluency was 
introduced by having participants typewrite essays using one 
hand (vs. standard typing). When essays were computationally 
analyzed, the results demonstrated that essays typewritten in 
the less fluent (or slower) condition were more lexically sophisti¬ 
cated (i.e., they exhibited increased lexical diversity and 
decreased word frequency). Medimorec and Risko (2016) inter¬ 
preted these results as consistent with the idea that typewriting 
may be too fluent (Heidegger, 1992; Norman 2002). For example, 
Norman (2002) suggested that handwriting encouraged more 
thoughtful writing compared to typewriting because the former 
was slower. Medimorec and Risko (2016) proposed a time- 
based account of the effects of disfluency on lexical sophistication 
whereby decreasing transcription fluency allowed more time for 
lexical processes, such as lexical access, to unfold. Such a mecha¬ 
nism is consistent with the general underpinnings of most theo¬ 
ries of lexical processing, which posit that accessing infrequent 
words takes more time than accessing more frequent words, both 
in speech and writing (Forster & Davis, 1984; Navarrete, Basagni, 
Alario, & Costa, 2006). For example, Crowe (1998) used a verbal 
fluency task to investigate lexical selection and found that partic¬ 
ipants first generated the more available frequent words followed 
by infrequent words. According to the time-based disfluency 
account, the critical variable in producing this effect on lexical 
sophistication should be the delay in transcription speed caused 
by disfluency. 
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While writing with one hand (relative to two) certainly slows 
down transcription, it also interferes with writing in other ways. 
For example, relatively skilled typewriting involves specific 
pairings of fingers and keyboard keys (Purcell, Napoliello, & 
Eden, 2011), and switching to one-handed typing would disrupt 
those mappings. This disruption could in theory influence lexical 
selection during writing. For example, there is evidence that indi¬ 
viduals exhibit different letter preferences when typing on a 
QWERTY keyboard across different input modalities depending 
on whether they use both hands or only one hand (Pelleg, Yom- 
Tov, & Gabrilovich, 2015). Thus the way that we type can influ¬ 
ence what we type. An effect on lexical sophistication would 
occur if lexical selection was influenced by the frequency of 
motor production (i.e., a bias towards selecting often typed 
words) and one-handed typing interfered with such an effect. In 
other words, the decreased lexical sophistication in the more flu¬ 
ent (or standard) typing condition could potentially be explained 
by a habitual combination of more rapid word access and the 
more automatized typing of those words (compared to infrequent 
words), with the latter being disrupted when we type one- 
handed. 

In order to test between the two accounts described above, it 
is necessary to And a manipulation that could slow down regular 
(both-handed) typing without disrupting the flnger-to-letter 
mappings. To this end, we had software developed that allowed 
us to introduce a delay between keypresses. Thus we could, 
relatively directly, control typing speed while individuals used 
their familiar two-handed typing (and thus maintained the same 
finger-to-letter relations across conditions). To our knowledge, 
this is the first time that the effects of such a manipulation 
have been reported. 

In addition to the two accounts described above, we also 
examined the idea that fluency effects in composition could 
result from effects on participants’ subjective task experience. 
Previous work has provided evidence that conscious experience 
of low effort or high speed, referred to as subjective fluency 
(Winkielman, Schwarz, Fazendeiro, & Reber, 2003), can influence 
cognitive processing. For example, subjective fluency can elicit a 
positive affective reaction (Schwarz & Gore, 1996) and facilitate 
creativity (Nadler, Rabi, & Minda, 2010). Moreover, individuals 
can perceive the same objective experiences as more or less 
fluent depending on previous experience and expectations 
(Whittlesea, 1993). Thus many fluency effects can involve a 
subjective fluency component. To test this notion, participants’ 
responses to the NASA Task Load Index (NASA-TLX; see Hart & 
Staveland, 1988), a multidimensional scale providing measures 
of subjective fluency, were examined. Finally, since increased 
lexical sophistication represents one of the predictors of better 
essay quality (Crossley & McNamara, 2011), we investigate 
whether disfluency effects on lexical sophistication extend to 
human essay quality judgements. 

2. Method 

2.3. Participants 

A total of 202 university students participated, but two partici¬ 
pants did not complete the study. Sample size was determined 
using the effect size of 0.40 (based on Medimorec & Risko, 2016), 
and power of 0.80 (Faul, Erdfelder, Buchner, & Lang, 2009). 

2.2. Design 

A 2 (standard vs. keyboard delay condition) between-subject 
design was used. 


2.3. Stimuli and apparatus 

The essays were written using a QWERTY keyboard and 
software that controlled the minimum time delay between 
keystrokes. Based on piloting, we set this minimum delay to 
100 ms. ^ Subjective fluency measures were collected using 
NASA-TLX, which has scales that range from -10 (i.e., low) to +10 
(i.e., high) for six dimensions: mental, physical, and temporal 
demand, performance, effort, and frustration. Keystroke activity 
was recorded using our delay application and the Inputlog 
key-logger (Leijten & Van Waes, 2013). 

2.4. Procedure 

Participants wrote a timed (50 min) argumentative essay 
regarding cellphone use in schools (Medimorec & Risko, 2016). Par¬ 
ticipants were asked to write at least 500 words and were 
informed that their essays would be graded. Participants were 
given a 3-min practice session to get familiarized with the key¬ 
board by typing a sentence. After the writing task, participants 
filled out the NASA-TLX scale. 

2.5. Measuring transcription fluency and linguistic features of essays 

Transcription fluency was determined by calculating times 
between consecutive lower case letters recorded by our delay 
application. We removed 1.42% of keystrokes within individual 
participants exceeding 2.5 SD of the mean (for 199 participants; 
one participant used all caps). 

Essays were analyzed using the Coh-Metrix text analyzer 
(McNamara, Graesser, McCarthy, & Cai, 2014). We also report 
corresponding language indices using the Tool for the 
Automatic Analysis of Lexical Sophistication (TAALES; Kyle & 
Crossley, 2015), and the Tool for the Automatic Analysis of 
Cohesion (TAACO; Crossley, Kyle, & McNamara, 2015). We focus 
on two relatively independent lexical sophistication indices 
affected by the transcription fluency manipulation in 
Medimorec and Risko (2016), specifically, lexical diversity and 
word frequency. 

2.5.1. Lexical diversity 

Lexical diversity is the range of vocabulary in a text. Coh-Metrix 
computes the type-token ratio (TTR; Templin, 1957), the measure 
of textual lexical diversity (MTLD; McCarthy & Jams, 2010), and 
vocd-D (Malvern, Richards, Chipere, & Duran, 2004). In general, 
these indices measure the ratio of unique words relative to the 
number of words in the text. Higher lexical diversity indicates 
greater lexical sophistication (McNamara et al., 2014). TTR results 
were also derived from TAACO. 

2.5.2. Word frequency 

Word frequency is a measure of how often individual words 
occur in the English language. Coh-Metrix derives the log- 
frequency for all words and the raw word frequency for content 
words from the CELEX database (Baayen, Piepenbrock, & 
Gulikers, 1996). Word frequencies from the SUBTLEXus 
(Brysbaert & New, 2009) and British National Corpus (BNC; BNC 
Consortium, 2007) databases are derived by TAALES. Texts with 
infrequent words are considered more lexically sophisticated 
(Crossley & McNamara, 2011). 


’ Note that this does not necessarily lead to a 100 ms decrease in typing speed 
because the majority of keypresses in regular typing already exceed 100 ms (~73% in 
our standard condition). Thus the introduced delay could be roughly conceptualized 
as a removal of all sub 100 ms interword keypresses from standard typing. 
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2.6. Essay grading 

Three trained raters independently and blind to condition 
scored essays using a 6-point rating scale (based on the ACT Writ¬ 
ing Test Scoring Rubric). Raters were trained to use the rubric by 
scoring two samples of argumentative essays from another corpus 
(JV = 30) until the averaged interrater reliability reached r>0.50. 
When the essays from the current study were scored, the interrater 
reliability was significant, r(198) = 0.56, p < 0.001. The mean score 
between the two raters with the highest correlation was used as 
the final score of each essay. If the differences between these 
two raters were 1, then the score closest to the third rater’s score 
was used. 

3. Results 

A series of one-way ANOVAs were performed with condition 
(standard vs. keyboard delay) as the factor and transcription flu¬ 
ency, lexical sophistication, subjective fluency, and essay quality 
as the dependent variables. 

3.1. Descriptive essay indices 

Condition affected transcription fluency such that it was 
more fluent in the standard condition, F(l,197) = 110.14, 
MSE = 2796.13, p < 0.001, d = 1.50. While there were more words 
typed in the standard condition, F(l, 198) = 6.35, MSF = 14329.49, 
p = 0.013, d = 0.36, essays in the delay condition contained longer 
words (letters and syllables per word), Fs>4.17, ps < 0.043, 
ds > 0.28. There were no other differences (see Table 1). 

3.2. Lexical sophistication 

Condition had a significant effect on type-token ratio, F( 1,198) 
= 9.70, MSE = 0.002, p = 0.002, d = 0.44, and vocd-D (marginally), F 
(1,198) = 3.29, MSE = 264.62, p = 0.071, d = 0.26, such that they 
were higher in the delay condition. There was no effect on the 
measure of textual lexical diversity, F(l,198) = 2.13, 


MSF = 239.96, p = 0.146, d = 0.21, though the pattern of means 
was in the same direction. Moreover, condition had an effect on 
both word frequency indices such that they were lower in the 
delay condition, log word frequency-all words, F(l,198) = 4.49, 
MSE = 0.01, p = 0.035, d = 0.30, raw word frequency-content words, 
F(l,198) = 4.74, MSE=0.02, p = 0.031, d = 0.31 (see Table 2). The 
results were similar when lexical diversity and word frequency 
from TAACO and TAALES were used (see Table 3). 

3.3. Subjective fluency 

Condition had an effect on two of the subjective fluency mea¬ 
sures, namely physical demand, F(l,198) = 12.34, MSE = 30.20, 
p = 0.001, d = 0.50, and frustration, F(l, 198) = 27.46, MS£ = 32.12, 
p < 0.001, d = 0.74, such that they were higher in the delay condi¬ 
tion. The other measures were unaffected, Fs <2.27, ps>0.13, 
ds < 0.22 (see Table 4). 

3.4. Essay quality 

There was no effect of condition on human-judged essay qual¬ 
ity, F(l,198) = 0.55, MS£ = 0.19, p = 0.458, d = 0.11. Graders rated 
the essays written with the standard keyboard (M = 4.24 
SD = 0.42) as similar to those written with the delayed keyboard 
(M = 4.19 SD = 0.45). Grades correlated positively with the number 
of words, r(198) = 0.41, p< 0.001. Controlling for the number of 
words, lower word frequency was related to higher quality, raw 
word frequency-content words 6 =-0.84, SF = 0.22, t(197) 
= -3.84, p< 0.001, rs=-0.24, log word frequency-all words, 
B = -1.16, S£ = 0.31, t(197) =-3.79, p< 0.001, rs=-0.24, whereas 
lexical diversity measures were not related to quality judgements, 
Bs < 0.58, ts < 0.81, ps < 0.44, tjS < 0.06. 

3.5. Regression analysis 

We performed a regression analysis to determine the extent to 
which the effects of disfluency on lexical indices are related to 
differences in the number of words and subjective fluency (since 


Table 1 

Means, Standard Deviations, and Cohen’s d (Cohen. 1988) of essay descriptive measures across the two conditions. 


Measure 

Standard 


Delay 


Difference 

Cohen’s d 

M 

SD 

M 

SD 

Transcription fluency 

223.37 

58.47 

302.05 

46.55 

1.50 

Number of words 

593.66 

138.56 

551.01 

97.27 

0.36 

Word length (letters) 

4.68 

0.24 

4.75 

0.25 

0.29 

Word length (syllables) 

1.49 

0.08 

1.51 

0.09 

0.31 

Sentence count 

25.91 

6.48 

25.39 

6.55 

0.08 

Words per sentence 

23.57 

4.90 

22.55 

4.92 

0.21 

Paragraph count 

4.12 

1.71 

4.15 

1.72 

0.02 


** p < 0.05 level. 

p< 0.001. 


Table 2 

Means, Standard Deviations, and Cohen’s d of lexical diversity and word frequency across the two conditions. Coh-Metrix indices. 


Measure Standard Delay Difference 



M 

SD 

M 

SD 

Cohen's d 

Type-token ratio 

0.41 

0.04 

0.43 

0.05 

0.44 

Measure of textual lexical diversity 

80.40 

14.43 

83.60 

16.49 

0.21 

vocd-D 

87.06 

14.81 

91.23 

17.61 

0.26 

Log frequency all words 

3.03 

0.09 

3.01 

0.08 

0.30 

Word frequency content words (Raw) 

2.37 

0.12 

2.33 

0.12 

0.31 


p < 0.10 level. 
** p < 0.05. 
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Table 3 

Means, Standard Deviations, and Cohen’s d of lexical diversity and word frequency across the two conditions, TAACO and TAALES indices. 


Measure 

Standard 

M 

SD 

Delay 

M 

SD 

Difference 

Cohen’s d 

Type-token ratio 

0.42 

0.04 

0.44 

0.05 

0.46 

Log frequency all words (SUBTLEXus) 

4.44 

0.14 

4.40 

0.13 

0.35 

Log frequency all words (BNC) 

4.91 

0.08 

4.87 

0.09 

0.37 

Log frequency content words (SUBTLEXus) 

3.77 

0.17 

3.71 

0.18 

0.35 

Log frequency content words (BNC) 

4.26 

0.10 

4.22 

0.12 

0.34 


"■ p < 0.05 level. 


Table 4 

Means, Standard Deviations, and Cohen's d of subjective fluency measures across the two conditions. 


Measure 

Standard 


Delay 


Difference 

Cohen’s d 

M 

SD 

M 

SD 

Mental demand 

0.16 

4.80 

1.10 

4.71 

0.20 

Physical demand 

-5.27 

4.81 

-2.54 

6.10 

0.50 

Temporal demand 

-1.82 

5.05 

-1.74 

5.43 

0.02 

Performance 

2.26 

4.51 

1.31 

4.96 

0.20 

Effort 

1.12 

4.13 

1.98 

3.96 

0.21 

Frustration 

-2.78 

5.48 

1.42 

5.85 

0.74 


" p < 0.05 level. 

"* p< 0.001. 


condition affected the number of words, physical demand, and 
frustration). We entered condition (0 = standard, 1 = delay) and 
the number of words as the independent variables, and raw 
frequency-content words or type-token ratio (indices with the lar¬ 
gest effect sizes) as the dependent variable. In the next step, we 
entered physical demand and frustration as additional indepen¬ 
dent variables. Effect sizes are semipartial correlations (ij). 

Condition had a significant effect on raw frequency-content 
words, B =-0.040, S£ = 0.02, t(197) =-2.27, p = 0.025, rs = -0.16, 
and type-token ratio, B = 0.012, SE = 0.01, t(197) = 2.13, p = 0.034, 
tj = 0.13, controlling for number of words. The effect remained sig¬ 
nificant with physical demand and frustration in the model for raw 
frequency-content words, B=-0.044, SE = 0.02, t(195) =-2.36, 
p = 0.019, Cs = -0.17, and type-token ratio (marginally), B = 0.010, 
SE = 0.01, t(195) = 1.74, p = 0.083, r, = 0.11. 


4. Discussion 

The results of the current study provide clear evidence against 
the mapping disruption account of disfluency on lexical sophistica¬ 
tion. In particular, by introducing a delay between consecutive key 
presses, we were able to slow typing without disrupting familiar 
finger-to-letter mappings. Despite the latter, we still observed a 
significant effect of disfluency on lexical sophistication. Specifi¬ 
cally, the essays written in the keyboard delay condition were 
more lexically diverse and used less frequent words. This result 
is consistent with the time-based account suggested by 
Medimorec and Risko (2016) wherein slowing typing provides 
additional time for lower frequency alternatives and more alterna¬ 
tives to be activated. 

The transcription fluency manipulation also influenced subjec¬ 
tive estimates of physical demand and frustration. Interestingly, 
the effect of disfluency on lexical diversity (marginally) and word 
frequency remained after controlling for these effects. Thus, while 
introducing a disfluency in transcription clearly has a marked 
effect on composers’ subjective experiences, there appears to be 
an effect of this disfluency on lexical sophistication that is indepen¬ 
dent of these effects. That said, it remains an interesting hypothesis 
to further pursue given that we only examined a limited number of 


subjective measures. In addition, we have focused on a specific set 
of lexical indices tied to lexical sophistication and there exists a 
large number of other lexical measures that could be influenced 
by introducing a transcription disfluency and the resulting subjec¬ 
tive experiences (in the supplementary materials, we provide addi¬ 
tional measures of lexical sophistication and measures from LIWC; 
Tausczik 8; Pennebaker, 2010). 

Lastly, the observed differences in lexical sophistication did not 
translate to an effect on subjective essay quality. There are a num¬ 
ber of potential reasons for this result. First, it is possible that our 
measure of grading was not sufficiently sensitive. For example, the 
reliability between essay graders was not particularly high. Second, 
it is possible that while lexical sophistication is related to essay 
quality, the magnitude of the effect of transcription disfluency on 
lexical sophistication (here d ~ 0.30) is not large enough to trans¬ 
late into human-graded essay quality. Lastly, it is also possible that 
the previously reported relation between lexical sophistication and 
essay quality (e.g., Crossley & McNamara, 2011) is confounded by 
some third variable. For example, rather than the use of lower fre¬ 
quency words causing higher essay scores, it might be that individ¬ 
uals who tend to use lower frequency words are superior on other 
aspects of writing that are more directly related to subjective essay 
quality. In the latter case, a manipulation that influences lexical 
sophistication but not these other aspects of writing would not 
be expected to translate into an effect on subjective essay quality. 

Tbe disfluency effects in written composition reported here 
diverge to some extent from previous research since we did not 
find a negative relation between transcription disfluency and writ¬ 
ing quality reported previously. For example, when participants in 
Olive, Alves, and Castro (2009) wrote essays using a disfluent cur¬ 
sive uppercase calligraphy, they produced essays that were judged 
to be of lesser quality to those written in participants’ own familiar 
calligraphy. Similarly, Alves, Castro, and Olive (2008) reported that 
slower typists produced texts that contained fewer different words 
compared to faster typists. Why the discrepancy? One hypothesis 
is that, unlike previous studies, the transcription disfluency manip¬ 
ulation in the current study did not introduce large working mem¬ 
ory demands. Consistent with this notion, there was no significant 
effect of condition on perceived mental demand or performance. In 
addition, in an unpublished experiment using a dual-task 
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technique, we found no difference between a standard typing con¬ 
dition and typing with a 100 ms deiay with respect to participants’ 
response time to a pseudorandom probe presented whiie they 
were writing.^ Not burdening working memory is important 
because iexicai seiection is iikeiy tied to the avaiiabiiity of working 
memory, indeed, previous research has demonstrated that increas¬ 
ing working memory ioad decreases verbai fluency (Rosen & Engie, 
1997). 

5. Conclusion 

The current study provided a test between a time-based and a 
mapping disruption account of the effects of disfluency on written 
composition. The resuits were consistent with the former account. 
That is, by providing additionai time for iexicai processes to unfoid, 
disfluency can increase iexicai sophistication in written composi¬ 
tion. in addition, we demonstrated effects of transcription disflu¬ 
ency on the subjective experience of that task (i.e., physicai 
demand, frustration), but these effects did not appear to expiain 
the effect of transcription fluency on iexicai sophistication. Finaiiy, 
the reported effects of disfluency on iexicai sophistication did not 
transiate to human-judged essay quaiity. 
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